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ABSTRACT 

The validity of inter pre ting teac hers' perceptions of 
their performance as an index of their actual performance was 
examined. Two matching instruments were constructed; each contained 
72 items in 6 categories of skills. One assessed teachers' 
perceptions of their competence on behaviorally stated generic 
teaching skills^ and the other assessed teachers' actual performance 
on the same skills. The items represented verbal information, concept 
ident if icaticn, or problem-solving skills that teachers need to 
perform the skill objectives; and skills that could realistically be 
assessed using pencil and paper questions* One hundred seventy^five 
classroom teachers were paid to participate in the study. Results 
showed that teachers' perception scores were significantly higher 
than their actual performance scores in all six content areas. Using 
teachers' perception scores to predict actual performance scores on 
teaching skills appeared to be an invalid practice; this finding held 
true for three different types of questions: recall of verbal 
information, concept identification, and problem- solving questions. 
It was also recommended that instructional needs of teacher education 
programs be determined by the teachers' actual performance rather 
than their perceived skills. (MV) 
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The performance-based or competency-based teacher education movement is becoming 
instrumental in the redefinition of goals for inservice and preservice teacher training institu- 
tions. More institutions are moving toward a belief that they should provide specific skills 
and knowledge wliich classroom teachers will need to effectively perform in today's schools. 
The redefinition of goals as performance, skills for classroom teachers has enabled teacher 
training institutions to evaluate current curriculum and instruction and begin to redefine 
both in terms of teachers' needs. This new emphasis on performance skills for classroom 
teachers is also being felt in teacher centers. 

Teacher centers are now being established by many states and local districts to be 
responsible for the inservice education of clsissroom teachers. Many are operated by local 

school districts while some are sponsored jointly by school distncts, universities, and state 

departments of education. Like many new organizations, teacher centers are undergoing tne 

processes of establishment, definition, and maintenance. The information needs for such 

early planning have been so great that in many districts special funding has been allocated 

for needs assessment studies to help gather this information. Legislation has been passed in 

many states to establish teacher education centers, and funds have been provided to enable 

needs assessments for program definition in teacher centers. In Florida, for example, special 

needs assessment programs are either planned, currently underway, or recently completed. 

One purpose for these needs assessment studies is to determine the skills and knowl- 
edge teachers should have and the skills and knowledge they actually do have. Skills which 
most teachers need but do not possess then become priority instruction for teacher centers 
or other agencies to provide. 

Information gained through these legislated needs assessment studies should provide 
input for decisions which will be made about teacher training by local schools, school 
districts, teacher centers, and teacher education programs in colleges and universities. 
Because of the importance of these decisions and implications they might have, assessment 
procedures and instruments used to gather information during these needs assessment 
projects should be studied. Considerable attention needs to be given to the types of deci- 
sions that can or should be made on the basis of assessment instruments used and results 
obtained. 



In a surv^ey of instruments currently used in these needs assessment studies, it was 
discovered that most assessment instruments were designed to measure the perceptions of 
teachers and other school personnel about the skills and knov/ledge they believe teachers 
should possess. Several studies also use perception instruments to survey teachers' perceived 
performance on specified skills. Items on these perception instruments are commonly oi the 
behaviorally ,sr.atcd skill statement type (Carey, 1974; Carey, 1975; Spindler, 1975; Florida 
State Department of Education, 1975). 

Based on information about teachers' perceived instructional needs and perceived 
competence to perform specified skills, teacher centers, universities, and state departments 
of education plan to obtain instruction which will be provided to the classroom teachers in 
the near future. 

The Problem 

The purpose of this study was to examine the validity of the current practice of inter- 
preting teachers' perceptions of their performance as an index of their actual performance 
on specified teaching skills. This is the procedure currently used by the state departments of 
education, universities, and teacher centers to assess the status of students and teachers on 
identified skills, to evaluate existing training programs, and to make decisions about forth- 
coming teacher training. 

Test validation defined by Cronbach (1971) is a process to determine the accuracy of 
inferences made from test scores; i.e., a process to examine the soundness of interpretation 
of a test. Regardless of how well a test was constructed or how representative the items of 
a test are to the content being measured, if the test is improperly interpreted, ttien it is 
considered invalid for the purposes intended. A validity study of any particular test does not 
stop with developmental, administrative, or scoring procedures. It must be determircd that 
test scores are being properly interpreted, and judgments, decisions, or predictions made 
from the scores are accurate. 

One type of validation, the validity of use and interpretation of tests, is defined by the 
American Psychological Association (1966) as criterion-related or predictive validation. The 
procedure for this-^type of validation requires the comparison of test ? ores with a variable 
considered to be a direct measure of the behavior in question. Actual performance scores are 
used as the criterion in this study to validate the use of teachers' perceptions of their perfor- 
mance on specified teaching tasks. 

A second method of validating the use of perception scores as accurate indicators of 
performance scores is to determine whether the same decisions would b^: made about 
inservice instruction needs if the decisions were based on either teachers' perceptions of 
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their performance or teachers' actual performance. This comparison of decisions that would 
be made would help determine how much confidence can be placed in inferences and 
decisions about classroom teachers' instructional needs which are based on measures of 
perception. 

At this point in time when many teacher centers are attempting to respond to the 
needs of their professional community, it is particularly appropriate to investigate the 
validity of the current practice of using teachers' perceptions of their performance as 
indicators of skills that need to be taught in inservice training programs. 

Specific hypotheses tested in the study were: 

1. There are no differences between teachers' perception scores and their performance 
scores on behaviorally stated generic teaching skills. 

2. There are no differences between teachers' perception and performance scores on 
verbal information questions. 

3. There arc no differences between teachers' perception and performance scores on 
concept identification questions. 

4. There are no differences between teachers' perception and performance scores on 
problem-solving questions. 

5. There is no difference in the selection of skills needed for inservice instruction 
when the selection is ba^ed on perception or performance scores. 

Methodology 

Instruments 

Two matching instruments were constructed. One was to assess teachers' perceptions 
of their competence on behaviorally stated generic teaching skills and the other was to assess 
teachers' actual performance on the same generic skills. 

The skills included in the perception and performance instruments were taken from 
instructional analyses of generic teaching skills that had been selected as important for 
classroom teachers to possess by over 90 percent of a group of educators representing 
classroom teachers, university faculty, and the Florida State Department of Education. 
Skills were selected from the instructional analyses for inclusion in the study if they: (a) 
represented either verbal information, concept identification, or problem-solving skills that 
teachers would need to perform the terminal objective in the skill hierarchy; and (b) repre- 
sented skills that could realistically be a';sessed using pencil and paper questions. 

The final perception and performance instruments each contained seventy-two items in 
six different categories of skills. These categories included skills in interacting with students, 
planning instruction, evaluating instruction and learning perfonnance, communicating with 
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peers and parents, classroom management, and foundutions of education. There were 12 
questions included on each of the instniinents for each of these six categories. Of the 12 
questions, four required the recall of verbal information, four required concept identifica- 
tion, and four were problem-solving questions. 

Procedures used to validate the instruments consisted of having a panel of appropriate 
judges determine whether: (a) the behavior stated in the skill statement on the perception 
test matched the behavior in the subskill in the original instructional analysis; (b) the 
question on the performance test measured the performance stated in the behavioral skill 
statement on the perception test; (c) the perceptual and performance skill items reflected 
skills classrooia teachers should possess; (d) th: questions represented skills that could be 
assessed using a paper and pencil instrument, and (e) th* 'vording of the questions was 
clear. A small group of teachers were asked to verbally interpret the instructions for each 
instrument and each item on each instrument to determine whether items and instructions 
were clear to sample members of the target group. 

Table 1 contains examples of behaviorally stated items from the perception instrument 
and matching test items from the performance test. 

Reliability detei-mined by coefficient alpha was .87 and .80 for the perception and the 
performance instruments respectively. 



Sample 

One hundred and seventy-iive classroon. teachers from Leon County, Florida partici- 
pated in the study. They represented the following groups: 



Age 



21-25 years 
19% 



26-^30 years 
23% 



31-40 years 
20% 



41+ years 
31% 



no indication 
7% 



Grade Preschool-2nd 3rd-5th f;th-8th 9th-12th no indication 

Currently 

Teaching 14%> 23% 27% 23%> 13% 



University 
Degree 



BS or BA 
58% 



i\lS or i\lA 
34%) 



no indication 
8% 



Procedures 

Teachers were paid co participate in the study which was scheduled at three local 
school sites immediately following the regular school day. Teachers were asked to: 

L Complete the perception instrument by reading each behaviorally stated teaching 
skill statement and determine whether or not (true or false) they could perform the stated 
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skiU. 

2. Complete the performance test a . r . dually providing the answers to each question. 

3, Identify themselves only by grade level taught, years of teaching experience, and 
current university degree. 

Nonparametric statistical procedures were used to analyze differences observed in 
teachers' perception and performance scores. 

Results 

Perception Scores and Performance Scores 

Teachers' perception scores were significantly higher (g_^.05) than their actual 
performance scores in all six content areas (Wilcoxon matched-pair signed-ranks est). 

Table 2 contains mean scores and standard deviations for perception and performance 
scores in all six content areas. 

Perception and Performance Scores Compared by Type of Question 

Teachers' scores on verbal information questions, concept identification questions, and 
problem-solving questions on the perception instrument were significantly higher (g_~ .05) 
than their scores on matching verbal information, concept identification, and problem- 
solving questions on the performance test in all six content areas (Wilcoxon matched-pairs 
signed-ranks test). 

Decisions Based on Perception Data Compared to Decisions Based on Performance Data 

The six content areas were prioritized with the lowest score receiving the highest 
priority for inservice training. Comparing the rank order of all six content areas for percep- 
tion and performance scores, the same decision about training priorities would be made one 
time out of six. A comparison of the rank order of the six skill areas on the perception and 
performance instruments is presented in Table 3. 

One other comparison was made of decisions based on perception and decisions based 
on performance. For this comparison, all item perception ;cores and all item performance 
scores were ranked from high to low. Items with the lowest scores thus represented skills 
for which training priority would be highest. Ten priority skills were then selected based on 
perception and ten were selected based on performance. A comparison of the two priority 
lists indicated that six of the skills were on both lists. 

When similar lists of the ten lowest priority skills were prepared, five of the skills were 
on both lists. 
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Conclusions 

The first type of validity studied was criterion or predictive validity in which a set of 
scores is compared against a variable considered to be a direct measure of the behavior in 
question. 

Using teachers' perception scores to predict actual performance scores on generic 
teaching skills appears to be an invaJid practice- This finding held true fcr three different 
types of questions, namely, recall of verbal information, concept identification, and prob- 
lem-solving questions. 

The second type of validity considered was the comparison of the priority decisions 
one would make using each type of data. The prioritization of instrucrional needs based on 
perception scores was different than the prioritization based on performance scores when 
compared by skill area and by individual skills. Based on the dissimilarity of training deci- 
sion that would have been made, it appears to be invalid to prioritize instructional needs 
solely on perception data. 

Discussion 

This study has demonstrated a need for the inclusion of performance data In the 
prioritization of inservice training needs. Though problems currently exist in measuring 
teachers' competence on specified teaching skills on a district-wide basis, the substitution 
of teachers' perception of their performance as a predictor for their actual performance is 
invalid. 

Some of the problems involved in performance testing to identify priority training 
needs are the time involved in administering tests, the time and personnel required to hand- 
score teachers' free response answers, and possibly the time required to observe teachers 
actually perform the skills in a real or simulated setting. An investigation should be conduc- 
ted into suitable procedures to select and test a representative sample of teachers in a 
district to determine their actual competence in selected skills. 

The acquisition of accurate data concerning teachers' competence on specified teaching 
skills is not the only ingredient that should be used to identify inservice training priorities. 
For example, teachers may all- fail to perform certain skills conectly, however, if no one 
in the districF considers those skills important for successful trar/ung, then the fact that 
teachers are not competent in them becomes unimportant. The opinions of educators 
concerning the skills teachers need and the skills they most want included in their inservice 
programs are ingredients that should be considered along with performance data to prior- 
itize skills for inservice training. 

Teachers' perceptions of what programs should be provided and the value of programs 
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that have been provided are useful and should be sought when planning, providing, and 
evaluating inservice education programs. However, these opinions should not be considered 
a substitute for actual preinstructional, postinstructional. -^nd follow-up assessment of teach- 
ers' a6i7zty to perform specified, basic teaching skills. 
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Table 1 

Sample iMatching Items from the Perception and the Performance Instruments 



Perception Item Performance Item 



1. Given a statement of the behavior in a 
verbal information learning task, you 
can describe the performance that 
would prove the student could perform 
the task (T) (F) 



1. Objective: The student will learn the 
names of three furbearing animals. 

Type of Learning: Verbal information 
(labels) aocep table proof the student 
has learned names: 



2. Given a diagram of a normal curve, you 
can identify the mean score. . . (T) (F) 



2. Identify the location of the mean score 
in the illustration below, and place the 
corresponding letter in the blank 




d e f g h i j k 



3. You can list different categories of 
information about students which you 
believe would be beneficial to teachers 
planning programs for new students 
' (T) (F) 



List three different categories of 
information you believe would be 
beneficial to teachers planning pro- 
grams for new students. 

a. ' 

b. . 

c. 



EKLC 



9 



Table 2 

Mean Scores and Standard Deviations for 
Perception and Performance Scores in All Six Content Areas 





Perception 


Perform: 


;tsce 




Instrument 


Test 




Content Area 


X 


SD 


X 


SD 


Interacting with students 


7.60* 


2.98 


6.40 


2.52 


Planning instruction 


8.90 


2.55 


5.32 


2.22 


Evaluating instruction 










and learning 


8.71 


3.19 


6.31 


2.93 


performance 










Communication with 
peers and parents 


8.5^- 


2.38 


2.54 


2.02 


Classroom management 


9.73 


2.41 


5.09 


2.49 


Foundations in 

education 


5.28 


4.33 


2.04 


1.92 



*Maximum score in any of the six content areas is 12. 
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Table 3 

Prioritization o f Instruct ir iua S^cd? by Skill Areas 
- Based on Perception s[y. P^-^rfomiance Data 



Rank 



Perception 



X 



Rank 



Performance 



X 



1 Foundations 



5.28 



1 Foundations 



2.04 



Interacting with 
students 



7.60 



Communication with 
2 2.54 
peers and parents 



Communication with 
peers and parents 



8.59 



3 Classroom management 5.09 



Evaluate instruction 
and performance 



8.71 



4 Planning instruction 5.32 



, . ' . Evaluate instruction 

0 Planmng msuuction 8.90 5 6.31 

and performance 



^, Interacting with 

6 Classroom management 9.73 6 6.40 

students 



